In H.264/AVC, one of the standard specifications for image encoding schemes, it is possible to use different quantization steps for each component of the orthogonal transform coefficients when quantizing image data in the High Profile or higher profile. A quantization step (or quantization scale) for each component of the orthogonal transform coefficients may be set on the basis of a quantization matrix (also called a scaling list) defined at the same size as the units of orthogonal transform, and a standard step value.
FIG. 38 illustrates four classes of default quantization matrices which are for intra prediction mode. The matrix SL2 is the default 4×4 quantization matrix for inter prediction mode. The matrix SL3 is the default 8×8 quantization matrix for intra prediction mode. The matrix SL4 is the default 8×8 quantization matrix for inter prediction mode. The user may also define one's own quantization matrix that differs from the default matrices illustrated in FIG. 38 in the sequence parameter set or the picture parameter set. Note that in the case where no quantization matrix is specified, a flat quantization matrix having an equal quantization step for all components may be used.
In High Efficiency Video Coding (HEVC), whose standardization is being advanced as a next-generation image encoding scheme to succeed H.264/AVC, there is introduced the concept of a coding unit (CU), which corresponds to a macroblock of the past (see Non-Patent Literature 1 below). The range of coding unit sizes is specified in the sequence parameter set as a pair of power-of-2 values called the largest coding unit (LCU) and the smallest coding unit (SCU). Additionally, SPLIT_FLAGs are used to designate a specific coding unit size within the range specified by the LCU and the SCU.
In HEVC, one coding unit may be split into one or more units of orthogonal transform, or in other words, one or more transform units (TUs). Any of 4×4, 8×8, 16×16, and 32×32 is usable as the transform unit size. Consequently, a quantization matrix may also be specified for each of these candidate transform unit sizes. Non-Patent Literature 2 below proposes specifying multiple quantization matrix candidates for one transform unit size in one picture, and adaptively selecting a quantization matrix for each block from the perspective of rate-distortion (RD) optimization.